A Framework for Conformance Testing of Systems Communicating through Rendezvous

In this papel; a formal framework is first proposed for conformance testing of communication systems, which are modeled by labeled transition systems (LTSs), in a systematic andoperational approach. In thisframework, test cases are limited to deterministic processes with finite behavior and state labels: testing is afinite set of experiments where every test case is parallelly composed with an implementation under test; observations are action sequences, executed during the testing, from which the test verdict is drawn directly. The fault model and fault coverage criteria are introduced to measure the effectiveness of testing. Afrerwards, based on this framework, for several common conformance relations, we present corresponding functions for the state labeling of test cases and upper bounds on the necessary sizes of test suites for obtaining complete fault coverage. One of the important issues in conformance testing of communication systems is to define a conceptual testing framework for systems communicating by rendezvous. Rendezvous does not distinguish explicitly between inputs and outputs, and communication between two processes occurs if both processes offer to interact on a particular action, and if the interaction takes place it occurs synchronously in both participating processes. One of the specification formalisms for such systems is labeled transition systems (LTSs); it also serves as a semantic model for various specification languages, e.g., LOTOS [21, CCS 1141, and CSP [91. Since there are various criteria for conformance of LTSs, from which a dozen of conformance relations are introduced [20], in order to facilitate test generation, a formal testing framework should be defined to answer the following questions: how are test cases structured for a given conformance relation, what constitute observations in testing and how is the verdict assigned? Moreover, this framework should also take into account the current practice, that is, conformance testing as a finite activity which should provide a well-defined confidence that the implementation under test conforms to its specification. Theories of conformance testing and methods for test derivation from LTSs have been developed in [3,4,20, 17, 21, 15, 19, 11, 7, 16, 181. A fundamental framework was introduced in [41. In this framework, theretum status of testing (successful or fail exit, deadlock, etc.) are considered as observations, and the verdict obtained from a given test case depends only on whether or not the observations obtained during testing of the IUT are a subset of the observations expected from the specification. However, intuitively, it is clear that the interaction sequences observed during testing may be important for defining the verdict, and should in general not be ignored. In fact, it is difficult for the framework of [4] to define the verdict for certain conformance relations, such as trace equivalence, nondeterminism reduction (conf plus trace equivalence) [6], and failure equivalence. Furthermore, since test cases with infinite and nondeterministic behavior are allowed, no attempt is made in this framework to describe how to obtain the verdict in an operational way through finite experiments. Another similar framework for conformance testing in the LTS formalism is drawn in [19] from the OS1 Conformance Testing Methodology and Framework. In this framework, states of test cases are directly labeled with verdicts, and the verdict assignment is obtained from the verdicts retumed during testing. Although this framework considers the action sequences observed during testing as observations, except for the conf relation, this framework does not answer how to structure test cases and assign the verdict for a given relation. Like the above framework, testing is treated as a correctness-proving process with respect to the given conformance relation. Therefore infinite testing is allowed and no fault coverage is considered. As a result, no estimation can be given for complexity of testing with guaranteed fault coverage. 0731-3071/96 $5.00